Field of the Invention
The present invention relates to an image processing apparatus, an image processing method and an image capture apparatus.
Description of the Related Art
Currently, in digital cameras (including digital cameras built in electronic devices such as smartphones and personal computers), a single-plane type color image sensor is generally used. The single-plane type color image sensor is provided with a mosaic color filter (hereinafter, simply referred to as a color filter), and generates a full-color image by a combination of the mosaic color filter and color interpolation processing (demosaicing).
There are many variations in the color components that constitute the color filter and in the arrangement thereof, but color filters with the primary color Bayer array are widely used. The primary color Bayer array is an array in which rows including alternately arranged red and green filters and rows including alternately arranged green and blue filters appear alternately in the column direction. In an image sensor provided with a color filter with the primary color Bayer array (hereinafter, referred to as a Bayer-type image sensor), each of the pixels outputs a signal corresponding to the luminance of red (R), green (G) or blue (B).
A full-color image is an image in which each of the pixels that constitute the image has the value of an RGB component. Therefore, in order to generate a full-color image from signals that can be obtained with a Bayer-type image sensor, it is necessary to generate two color component values for each pixel by demosaicing. Various algorithms have been suggested regarding demosaicing in order to obtain a high quality image, but interpolation is performed based on signals at other positions, and thus there is the possibility that color moiré and a pseudo color will occur.
Meanwhile, a method for obtaining a high-quality full-color image using a plurality of image sensors has been suggested. Japanese Patent Laid-Open No. 2009-105984 discloses a technique for obtaining a moving image that has a high resolution and a high frame rate using two image sensors having the same structure as a color image sensor (640 horizontal RGB pixels) and a monochromatic image sensor (1920 horizontal monochromatic pixels). Specifically, after capturing alternately with the two image sensors, frame interpolation is performed on moving images obtained by the image sensors such that the moving images are converted into moving images having a frame rate twice the frame rate at the time of the capturing. Two frame images corresponding to the same time are combined so as to generate a color moving image having a high luminance resolution and a frame rate twice the frame rate at the time of the capturing.
Moreover, in Japanese Patent Laid-Open No. 2001-169301, two full-color images are generated using first and second Bayer-type image sensors arranged so as to be shifted by ½ pixel from each other, and the pixels are arranged in a checkered pattern, whereby image data having a higher resolution than image data generated by separately using the image sensors is generated.
According to the technique of Japanese Patent Laid-Open No. 2009-105984, luminance information and color information are alternately obtained, and thus timings for obtaining the information are different, thereby causing deterioration in the image quality of composite images especially in moving image capturing.
Moreover, in Japanese Patent Laid-Open No. 2001-169301, the images obtained with the two image sensors are combined after color interpolation processing, and an improvement in the accuracy of color interpolation processing is not taken into consideration.